Fast Stochastic Optimization Algorithms for ML
نویسنده
چکیده
For regression, y is real valued, L is the often squared loss (but not always; for example see Least Absolute Deviation regression) and w is the best linear fit to the data. For binary classification, y is binary, L is often the 0/1 loss and w is the best hyperplane separating the two sets of samples. Since the 0/1 loss is nonconvex, we often use convex upper bounds for the 0/1 loss examples of such surrogate losses include the logistic, hinge, exponential and squared losses.
منابع مشابه
A Bi-objective Stochastic Optimization Model for Humanitarian Relief Chain by Using Evolutionary Algorithms
Due to the increasing amount of natural disasters such as earthquakes and floods and unnatural disasters such as war and terrorist attacks, Humanitarian Relief Chain (HRC) is taken into consideration of most countries. Besides, this paper aims to contribute humanitarian relief chains under uncertainty. In this paper, we address a humanitarian logistics network design problem including local dis...
متن کاملThree Hybrid Metaheuristic Algorithms for Stochastic Flexible Flow Shop Scheduling Problem with Preventive Maintenance and Budget Constraint
Stochastic flexible flow shop scheduling problem (SFFSSP) is one the main focus of researchers due to the complexity arises from inherent uncertainties and also the difficulty of solving such NP-hard problems. Conventionally, in such problems each machine’s job process time may encounter uncertainty due to their relevant random behaviour. In order to examine such problems more realistically, fi...
متن کاملDesigning Stochastic Cell Formation Problem Using Queuing Theory
This paper presents a new nonlinear mathematical model to solve a cell formation problem which assumes that processing time and inter-arrival time of parts are random variables. In this research, cells are defined as a queue system which will be optimized via queuing theory. In this queue system, each machine is assumed as a server and each part as a customer. The grouping of machines and parts...
متن کاملBalancing the Communication Load of Asynchronously Parallelized Machine Learning Algorithms
Stochastic Gradient Descent (SGD) is the standard numerical method used to solve the core optimization problem for the vast majority of machine learning (ML) algorithms. In the context of large scale learning, as utilized by many Big Data applications, efficient parallelization of SGD is in the focus of active research. Recently, we were able to show that the asynchronous communication paradigm...
متن کاملA FAST FUZZY-TUNED MULTI-OBJECTIVE OPTIMIZATION FOR SIZING PROBLEMS
The most recent approaches of multi-objective optimization constitute application of meta-heuristic algorithms for which, parameter tuning is still a challenge. The present work hybridizes swarm intelligence with fuzzy operators to extend crisp values of the main control parameters into especial fuzzy sets that are constructed based on a number of prescribed facts. Such parameter-less particle ...
متن کاملA Hybrid Optimization Algorithm for Learning Deep Models
Deep learning is one of the subsets of machine learning that is widely used in Artificial Intelligence (AI) field such as natural language processing and machine vision. The learning algorithms require optimization in multiple aspects. Generally, model-based inferences need to solve an optimized problem. In deep learning, the most important problem that can be solved by optimization is neural n...
متن کامل